Skip to content

Save/load snapshot from disk (oci format)#1465

Open
ludfjig wants to merge 4 commits into
hyperlight-dev:mainfrom
ludfjig:disk_snapshot_oci
Open

Save/load snapshot from disk (oci format)#1465
ludfjig wants to merge 4 commits into
hyperlight-dev:mainfrom
ludfjig:disk_snapshot_oci

Conversation

@ludfjig

@ludfjig ludfjig commented May 20, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Adds save/load of Snapshot to disk as an OCI Image Layout. Public API:

  • Snapshot::to_oci(path, tag)
  • Snapshot::from_oci(path, tag) (verifies sha256 on every blob)
  • Snapshot::from_oci_unchecked(path, tag) (unsafe because skips digest checks, not unsafe in rust UB sense)

Known limitations

  • Core dumps from a snapshot-loaded sandbox lack binary_path and AT_ENTRY for
    Call snapshots. mem_profile lacks accurate traces. Fixing either requires
    extending the on-disk format.

  • max_guest_log_level is not plumbed through snapshot load. It is also
    intrinsically ineffective for Call snapshots and should be addressed
    separately.

  • The backing OCI directory must not be modified, truncated, renamed over, or
    deleted for the lifetime of a loaded Snapshot or any MultiUseSandbox
    built from it. On Linux this is unenforced. On Windows the OS refuses the
    operation with ERROR_USER_MAPPED_FILE (1224). Firecracker has the same
    constraint:

    The memory file (pointed by backend_path when using File backend type,
    or pointed by mem_file_path) must be considered immutable from
    Firecracker and host point of view. It backs the guest OS memory for read
    access through the page cache. External modification to this file corrupts
    the guest memory and leads to undefined behavior.

    firecracker docs

Future work

Typed error variants. TSC and rand seeding capture. Fuzz target for from_oci.
CoW overlay layers. Cross-hypervisor portability via sregs normalisation. Huge
page support (MAP_HUGETLB)

@ludfjig ludfjig force-pushed the disk_snapshot_oci branch 10 times, most recently from 84d17dd to c4aec72 Compare June 3, 2026 22:27
@ludfjig ludfjig force-pushed the disk_snapshot_oci branch from c4aec72 to 012bd11 Compare June 8, 2026 17:52
@simongdavies simongdavies mentioned this pull request Jun 8, 2026
Open
@ludfjig ludfjig force-pushed the disk_snapshot_oci branch 3 times, most recently from 4cb8124 to 7c8175b Compare June 9, 2026 16:37
@ludfjig ludfjig marked this pull request as ready for review June 9, 2026 16:38
Copilot AI review requested due to automatic review settings June 9, 2026 16:38
Copilot AI reviewed Jun 9, 2026
View reviewed changes

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an OCI Image Layout–backed on-disk format for Snapshot, enabling saving a sandbox snapshot to disk and loading it back (with optional digest verification) as a building block for snapshot-driven sandbox creation (per the dependency on #1459).

Changes:

  • Implement Snapshot::to_oci, Snapshot::from_oci, and Snapshot::from_oci_unchecked using OCI Image Layout + sha256-addressed blobs.
  • Add serde-encoded snapshot config schema (arch/hypervisor/ABI gating, entrypoint/sregs, layout, host function signatures) plus extensive validation + tests.
  • Extend benchmarks and docs to cover snapshot file save/load paths and usage.

Reviewed changes

Copilot reviewed 12 out of 14 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
src/hyperlight_host/src/sandbox/snapshot/mod.rs Wires in the new snapshot OCI file backend modules.
src/hyperlight_host/src/sandbox/snapshot/file/mod.rs Implements OCI layout writer/loader for snapshots (manifest/config/blob handling).
src/hyperlight_host/src/sandbox/snapshot/file/config.rs Defines and validates the JSON config schema for OCI snapshot artifacts.
src/hyperlight_host/src/sandbox/snapshot/file/digest.rs Adds sha256 digest helpers and verification for blobs.
src/hyperlight_host/src/sandbox/snapshot/file/fsutil.rs Adds bounded file read + atomic write + content-addressed blob write helpers.
src/hyperlight_host/src/sandbox/snapshot/file/media_types.rs Defines versioned media types and ABI version constant for snapshot artifacts.
src/hyperlight_host/src/sandbox/snapshot/file_tests.rs Adds comprehensive unit/integration tests for the on-disk OCI snapshot format.
src/hyperlight_host/src/sandbox/initialized_multi_use.rs Updates docs to show creating a sandbox from an on-disk-loaded snapshot.
src/hyperlight_host/src/mem/shared_mem.rs Adjusts Linux file-backed mapping protection flags for MSHV requirements.
src/hyperlight_host/Cargo.toml Adds dependencies needed for OCI + hashing + serde config.
src/hyperlight_host/benches/benchmarks.rs Adds benchmarks for sandbox-from-snapshot and snapshot file save/load/cold-start paths.
docs/snapshot-oci-format.md Documents the OCI Image Layout snapshot on-disk format and semantics.
CHANGELOG.md Notes the new snapshot persistence APIs.
Cargo.lock Locks newly added transitive dependencies.

Comment thread src/hyperlight_host/src/sandbox/snapshot/file/mod.rs
use crate::hypervisor::regs::CommonSpecialRegisters;
use crate::mem::layout::SandboxMemoryLayout;
use crate::mem::memory_region::MemoryRegionFlags;
use crate::mem::shared_mem::{ReadonlySharedMemory, SharedMemory};

@ludfjig ludfjig Jun 9, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is bull

Comment thread src/hyperlight_host/src/mem/shared_mem.rs
Comment on lines 1655 to 1658
// 2. Overlay the file content on the middle slot with
// `MAP_FIXED`. The guest maps these pages READ|EXECUTE,
// so the host VMA is read-only. `MAP_PRIVATE` keeps the
// mapping detached from the underlying file.

@ludfjig ludfjig Jun 9, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a comment right there

Comment thread src/hyperlight_host/Cargo.toml
Comment on lines +51 to +55
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
elfcore = { version = "2.0", optional = true }
uuid = { version = "1.23.2", features = ["v4"] }
oci-spec = { version = "0.8", default-features = false, features = ["image"] }

@ludfjig ludfjig Jun 9, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

image

@ludfjig ludfjig added the kind/enhancement For PRs adding features, improving functionality, docs, tests, etc. label Jun 9, 2026
@ludfjig ludfjig force-pushed the disk_snapshot_oci branch from 7c8175b to a37f54e Compare June 9, 2026 17:32
@ludfjig
Map snapshot pages writable on MSHV
be93bab 
Signed-off-by: Ludvig Liljenberg <4257730+ludfjig@users.noreply.github.com>
@ludfjig ludfjig force-pushed the disk_snapshot_oci branch 2 times, most recently from 517f50d to 54107fb Compare June 9, 2026 18:40
ludfjig added 2 commits June 9, 2026 12:05
@ludfjig
Add OCI snapshot persistence
2819ff1 
Signed-off-by: Ludvig Liljenberg <4257730+ludfjig@users.noreply.github.com>
@ludfjig
Add tests for OCI snapshot persistence
79f3823 
Signed-off-by: Ludvig Liljenberg <4257730+ludfjig@users.noreply.github.com>
@ludfjig ludfjig force-pushed the disk_snapshot_oci branch from 54107fb to f270a7e Compare June 9, 2026 19:05
@ludfjig
Add OCI snapshot benchmarks
8a542b8 
Signed-off-by: Ludvig Liljenberg <4257730+ludfjig@users.noreply.github.com>
@ludfjig ludfjig force-pushed the disk_snapshot_oci branch from f270a7e to 8a542b8 Compare June 9, 2026 20:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments

@danbugs danbugs Awaiting requested review from danbugs danbugs is a code owner

@devigned devigned Awaiting requested review from devigned devigned is a code owner

@dblnz dblnz Awaiting requested review from dblnz dblnz is a code owner

@jsturtevant jsturtevant Awaiting requested review from jsturtevant jsturtevant is a code owner

@jprendes jprendes Awaiting requested review from jprendes jprendes is a code owner

@syntactically syntactically Awaiting requested review from syntactically syntactically is a code owner

@simongdavies simongdavies Awaiting requested review from simongdavies simongdavies is a code owner

@andreiltd andreiltd Awaiting requested review from andreiltd andreiltd is a code owner

@squillace squillace Awaiting requested review from squillace squillace is a code owner

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

kind/enhancement For PRs adding features, improving functionality, docs, tests, etc.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@ludfjig